Tracing Quickstart - Weights & Biases

Calls Calls are the fundamental building block in Weave. They represent a single execution of a function, including:

Inputs (arguments)
Outputs (return value)
Metadata (duration, exceptions, LLM usage, etc.)

Calls are similar to spans in the OpenTelemetry data model. A Call can:

Belong to a Trace (a collection of calls in the same execution context)
Have parent and child Calls, forming a tree structure

Creating Calls

There are three main ways to create Calls in Weave:

1. Automatic tracking of LLM libraries

Weave automatically tracks calls to common LLM libraries like openai, anthropic, cohere, and mistral. Simply call weave.init('project_name') at the start of your program:

You can control Weave’s default tracking behavior using the autopatch_settings argument in weave.init.

showLineNumbers

import weave

from openai import OpenAI
client = OpenAI()

# Initialize Weave Tracing
weave.init('intro-example')

response = client.chat.completions.create(
    model="gpt-4",
    messages=[
        {
            "role": "user",
            "content": "How are you?"
        }
    ],
    temperature=0.8,
    max_tokens=64,
    top_p=1,
)

Summary

You can store metrics or other post-call values in the summary dictionary of a Call. Modify call.summary during execution and any values you add will be merged with Weave’s computed summary data when the call finishes.

2. Decorating and wrapping functions

However, often LLM applications have additional logic (such as pre/post processing, prompts, etc.) that you want to track.

Weave allows you to manually track these calls using the @weave.op decorator. For example:

showLineNumbers

import weave

# Initialize Weave Tracing
weave.init('intro-example')

# Decorate your function
@weave.op
def my_function(name: str):
    return f"Hello, {name}!"

# Call your function -- Weave will automatically track inputs and outputs
print(my_function("World"))

You can also track methods on classes.

Trace sync & async generator functions

Weave supports tracing both sync and async generator functions, including deeply nested patterns.

Since generators yield values lazily, the outputs are only logged when the generator is fully consumed (e.g., by converting it to a list). To ensure outputs are captured in the trace, fully consume the generator (e.g., by using list()).

showLineNumbers

from typing import Generator
import weave

weave.init("my-project")

# This function uses a simple sync generator.
# Weave will trace the call and its input (`x`), 
# but output values are only captured once the generator is consumed (e.g., via `list()`).
@weave.op
def basic_gen(x: int) -> Generator[int, None, None]:
    yield from range(x)

# A normal sync function used within the generator pipeline.
# Its calls are also traced independently by Weave.
@weave.op
def inner(x: int) -> int:
    return x + 1

# A sync generator that calls another traced function (`inner`).
# Each yielded value comes from a separate traced call to `inner`.
@weave.op
def nested_generator(x: int) -> Generator[int, None, None]:
    for i in range(x):
        yield inner(i)

# A more complex generator that composes the above generator.
# Tracing here produces a hierarchical call tree:
# - `deeply_nested_generator` (parent)
#   - `nested_generator` (child)
#     - `inner` (grandchild)
@weave.op
def deeply_nested_generator(x: int) -> Generator[int, None, None]:
    for i in range(x):
        for j in nested_generator(i):
            yield j

# The generator must be *consumed* for Weave to capture outputs.
# This is true for both sync and async generators.
res = deeply_nested_generator(4)
list(res)  # Triggers tracing of all nested calls and yields

Getting a handle to the call object during execution

Sometimes it is useful to get a handle to the Call object itself. You can do this by calling the op.call method, which returns both the result and the Call object. For example:

showLineNumbers

result, call = my_function.call("World")

Then, call can be used to set / update / fetch additional properties (most commonly used to get the ID of the call to be used for feedback).

If your op is a method on a class, you need to pass the instance as the first argument to the op (see example below).

showLineNumbers

# Notice that we pass the `instance` as the first argument.
print(instance.my_method.call(instance, "World"))

showLineNumbers

import weave

# Initialize Weave Tracing
weave.init("intro-example")

class MyClass:
    # Decorate your method
    @weave.op
    def my_method(self, name: str):
        return f"Hello, {name}!"

instance = MyClass()

# Call your method -- Weave will automatically track inputs and outputs
instance.my_method.call(instance, "World")

Call display name

Sometimes you may want to override the display name of a call. You can achieve this in one of four ways:

Change the display name at the time of calling the op:

showLineNumbers

result = my_function("World", __weave={"display_name": "My Custom Display Name"})

Using the __weave dictionary sets the call display name which will take precedence over the Op display name.

Change the display name on a per-call basis. This uses the Op.call method to return a Call object, which you can then use to set the display name using Call.set_display_name.

showLineNumbers

result, call = my_function.call("World")
call.set_display_name("My Custom Display Name")

Change the display name for all Calls of a given Op:

showLineNumbers

@weave.op(call_display_name="My Custom Display Name")
def my_function(name: str):
    return f"Hello, {name}!"

The call_display_name can also be a function that takes in a Call object and returns a string. The Call object will be passed automatically when the function is called, so you can use it to dynamically generate names based on the function’s name, call inputs, fields, etc.

One common use case is just appending a timestamp to the function’s name.

from datetime import datetime

@weave.op(call_display_name=lambda call: f"{call.func_name}__{datetime.now()}")
def func():
    return ...

You can also log custom metadata using .attributes

def custom_attribute_name(call):
    model = call.attributes["model"]
    revision = call.attributes["revision"]
    now = call.attributes["date"]

    return f"{model}__{revision}__{now}"

@weave.op(call_display_name=custom_attribute_name)
def func():
    return ...

with weave.attributes(
    {
        "model": "finetuned-llama-3.1-8b",
        "revision": "v0.1.2",
        "date": "2024-08-01",
    }
):
    func()  # the display name will be "finetuned-llama-3.1-8b__v0.1.2__2024-08-01"

    with weave.attributes(
        {
            "model": "finetuned-gpt-4o",
            "revision": "v0.1.3",
            "date": "2024-08-02",
        }
    ):
        func()  # the display name will be "finetuned-gpt-4o__v0.1.3__2024-08-02"

Technical Note: “Calls” are produced by “Ops”. An Op is a function or method that is decorated with @weave.op. By default, the Op’s name is the function name, and the associated calls will have the same display name. The above example shows how to override the display name for all Calls of a given Op. Sometimes, users wish to override the name of the Op itself. This can be achieved in one of two ways:

Set the name property of the Op before any calls are logged

showLineNumbers

my_function.name = "My Custom Op Name"

Set the name option on the op decorator

showLineNumbers

@weave.op(name="My Custom Op Name)

Attributes

When calling tracked functions, you can add additional metadata to the call by using weave.attributes context manager. In the example below, we add an env attribute to the call specified as 'production'.

showLineNumbers

# ... continued from above ...

# Add additional attributes to the call
with weave.attributes({'env': 'production'}):
    print(my_function.call("World"))

call.attributes cannot be modified once the call starts. Use this context manager to set any metadata before invoking the op.

Trace parallel (multi-threaded) function calls

By default, parallel calls all show up in Weave as separate root calls. To get correct nesting under the same parent op, use ThreadPoolExecutor.The following code sample demonstrates the use of ThreadPoolExecutor. The first function, func, is a simple op that takes x and returns x+1. The second function, outer, is another op that accepts a list of inputs. Inside outer, the use of ThreadPoolExecutor and exc.map(func, inputs) means that each call to func still carries the same parent trace context.

import weave

@weave.op
def func(x):
    return x+1

@weave.op
def outer(inputs):
    with weave.ThreadPoolExecutor() as exc:
        exc.map(func, inputs)

# Update your Weave project name  
client = weave.init('my-weave-project')
outer([1,2,3,4,5])

In the Weave UI, this produces a single parent call with five nested child calls, so that you get a fully hierarchical trace even though the increments run in parallel. The Trace UI, showing a single parent call for outer, with five nested child calls.

The Trace UI, showing a single parent call for outer, with five nested child calls.

3. Manual Call tracking

You can also manually create Calls using the API directly.

showLineNumbers

import weave

# Initialize Weave Tracing
client = weave.init('intro-example')

def my_function(name: str):
    # Start a call
    call = client.create_call(op="my_function", inputs={"name": name})

    # ... your function code ...

    # End a call
    client.finish_call(call, output="Hello, World!")

# Call your function
print(my_function("World"))

4. Track class and object methods

You can also track class and object methods.

Track any method on a class using weave.op.

showLineNumbers

import weave

# Initialize Weave Tracing
weave.init("intro-example")

class MyClass:
    # Decorate your method
    @weave.op
    def my_method(self, name: str):
        return f"Hello, {name}!"

instance = MyClass()

# Call your method -- Weave will automatically track inputs and outputs
print(instance.my_method("World"))

Viewing Calls

To view a call in the web app:

Navigate to your project’s “Traces” tab
Find the call you want to view in the list
Click on the call to open its details page

The details page will show the call’s inputs, outputs, runtime, and any additional metadata. View Call in Web App

Updating Calls

Calls are mostly immutable once created, however, there are a few mutations which are supported:

Set Display Name
Add Feedback
Delete a Call

All of these mutations can be performed from the UI by navigating to the call detail page: Update Call in Web App

Set display name

In order to set the display name of a call, you can use the Call.set_display_name method.

showLineNumbers

import weave

# Initialize the client
client = weave.init("your-project-name")

# Get a specific call by its ID
call = client.get_call("call-uuid-here")

# Set the display name of the call
call.set_display_name("My Custom Display Name")

Add feedback

Please see the Feedback Documentation for more details.

Delete a Call

To delete a Call using the Python API, you can use the Call.delete method.

showLineNumbers

import weave

# Initialize the client
client = weave.init("your-project-name")

# Get a specific call by its ID
call = client.get_call("call-uuid-here")

# Delete the call
call.delete()

Delete multiple Calls

To delete batches of Calls using the Python API, pass a list of Call IDs to delete_calls().

showLineNumbers

import weave

# Initialize the client
client = weave.init("my-project")

# Get all calls from client 
all_calls = client.get_calls()

# Get list of first 1000 Call objects
first_1000_calls = all_calls[:1000]

# Get list of first 1000 Call IDs
first_1000_calls_ids = [c.id for c in first_1000_calls]

# Delete first 1000 Call objects by ID
client.delete_calls(call_ids=first_1000_calls_ids)

Querying and exporting Calls

The /calls page of your project (“Traces” tab) contains a table view of all the Calls in your project. From there, you can:

Sort
Filter
Export

The Export Modal (shown above) allows you to export your data in a number of formats, as well as shows the Python & CURL equivalents for the selected calls! The easiest way to get started is to construct a view in the UI, then learn more about the export API via the generated code snippets.

To fetch calls using the Python API, you can use the client.get_calls method:

import weave

# Initialize the client
client = weave.init("your-project-name")

# Fetch calls
calls = client.get_calls(filter=...)

Call schema

Please see the schema for a complete list of fields.

Property	Type	Description
id	string (uuid)	Unique identifier for the call
project_id	string (optional)	Associated project identifier
op_name	string	Name of the operation (can be a reference)
display_name	string (optional)	User-friendly name for the call
trace_id	string (uuid)	Identifier for the trace this call belongs to
parent_id	string (uuid)	Identifier of the parent call
started_at	datetime	Timestamp when the call started
attributes	Dict[str, Any]	User-defined metadata about the call (read-only during execution)
inputs	Dict[str, Any]	Input parameters for the call
ended_at	datetime (optional)	Timestamp when the call ended
exception	string (optional)	Error message if the call failed
output	Any (optional)	Result of the call
summary	Optional[SummaryMap]	Post-execution summary information. You can modify this during execution to record custom metrics.
wb_user_id	Optional[str]	Associated Weights & Biases user ID
wb_run_id	Optional[str]	Associated Weights & Biases run ID
deleted_at	datetime (optional)	Timestamp of call deletion, if applicable

The table above outlines the key properties of a Call in Weave. Each property plays a crucial role in tracking and managing function calls:

The id, trace_id, and parent_id fields help in organizing and relating calls within the system.
Timing information (started_at, ended_at) allows for performance analysis.
The attributes and inputs fields provide context for the call. Attributes are frozen once the call starts, so set them before invocation with weave.attributes. output and summary capture the results, and you can update summary during execution to log additional metrics.
Integration with Weights & Biases is facilitated through wb_user_id and wb_run_id.

This comprehensive set of properties enables detailed tracking and analysis of function calls throughout your project. Calculated Fields:

Cost
Duration
Status

Saved views

You can save your Trace table configurations, filters, and sorts as saved views for quick access to your preferred setup. You can configure and access saved views via the UI and the Python SDK. For more information, see Saved Views.

View a W&B run in the Traces table

With Weave, you can trace function calls in your code and link them directly to the W&B runs in which they were executed. When you trace a function with @weave.op() and call it inside a wandb.init() context, Weave automatically associates the trace with the W&B run. Links to any associated runs are shown in the Traces table.

Python example

The following Python code shows how traced operations are linked to W&B runs when executed inside a wandb.init() context. These traces appear in the Weave UI and are associated with the corresponding run.

import wandb
import weave

def example_wandb(projname):
    # Split projname into entity and project
    entity, project = projname.split("/", 1)

    # Initialize Weave context for tracing
    weave.init(projname)

    # Define a traceable operation
    @weave.op()
    def say(message: str) -> str:
        return f"I said: {message}"

    # First W&B run
    with wandb.init(
        entity=entity,
        project=project,
        notes="Experiment 1",
        tags=["baseline", "paper1"],
    ) as run:
        say("Hello, world!")
        say("How are you!")
        run.log({"messages": 2})

    # Second W&B run
    with wandb.init(
        entity=entity,
        project=project,
        notes="Experiment 2",
        tags=["baseline", "paper1"],
    ) as run:
        say("Hello, world from experiment 2!")
        say("How are you!")
        run.log({"messages": 2})

if __name__ == "__main__":
    # Replace this with your actual W&B username/project
    example_wandb("your-username/your-project")

To use the code sample:

In the terminal, install dependencies:
```
pip install wandb weave
```
Log in to W&B:
```
wandb login
```
In the script, replace your-username/your-project with your actual W&B entity/project.
Run the script:
```
python weave_trace_with_wandb.py
```
Visit https://weave.wandb.ai and select your project.
In the Traces tab, view the trace output. Links to any associated runs are shown in the Traces table.

Configure autopatching

By default, Weave automatically patches and tracks calls to common LLM libraries like openai, anthropic, cohere, and mistral. You can control this behavior using the autopatch_settings argument in weave.init.

Disable all autopatching

showLineNumbers

weave.init(..., autopatch_settings={"disable_autopatch": True})

Disable a specific integration

showLineNumbers

weave.init(..., autopatch_settings={"openai": {"enabled": False}})

Post-process inputs and outputs

You can also customize how post-process inputs and outputs (e.g. for PII data) are handled during autopatching:

showLineNumbers

def redact_inputs(inputs: dict) -> dict:
    if "email" in inputs:
        inputs["email"] = "[REDACTED]"
    return inputs

weave.init(
    ...,
    autopatch_settings={
        "openai": {
            "op_settings": {
                "postprocess_inputs": redact_inputs,
            }
        }
    }
)

For more details, see How to use Weave with PII data.

FAQs

How do I stop large traces from being truncated?

For more information, see Trace data is truncated in the Troubleshooting guide.

How do I disable tracing?

Environment variable

In situations where you want to unconditionally disable tracing for the entire program, you can set the environment variable WEAVE_DISABLED=true.

Client initialization

Sometimes, you may want to conditionally enable tracing for a specific initialization based on some condition. In this case, you can initialize the client with the disabled flag in init settings.

import weave

# Initialize the client
client = weave.init(..., settings={"disabled": True})

Context manager

Finally, you may want to conditionally disable tracing for a single function based on some application logic. In this case, you can use the context manager with set_tracing_enabled(False) which can be imported from weave.trace.context.call_context.

import weave
from weave.trace.context.call_context import set_tracing_enabled

client = weave.init(...)

@weave.op
def my_op():
    ...

with set_tracing_enabled(False):
    my_op()

How do I capture information about a Call?

Typically you would call an op directly:

@weave.op
def my_op():
    ...

my_op()

However, you can also get access to the call object directly by invoking the call method on the op:

@weave.op
def my_op():
    ...

output, call = my_op.call()

From here, the call object will have all the information about the call, including the inputs, outputs, and other metadata.

Introduction

Iteration

Evaluation

Productionization

Integrations

Enterprise

Tools & Resources

​Creating Calls

​1. Automatic tracking of LLM libraries

​Summary

​2. Decorating and wrapping functions

​Trace sync & async generator functions

​Getting a handle to the call object during execution

​Call display name

​Attributes

​Trace parallel (multi-threaded) function calls

​3. Manual Call tracking

​4. Track class and object methods

​Viewing Calls

​Updating Calls

​Set display name

​Add feedback

​Delete a Call

​Delete multiple Calls

​Querying and exporting Calls

​Call schema

​Saved views

​View a W&B run in the Traces table

​Python example

​Configure autopatching

​Disable all autopatching

​Disable a specific integration

​Post-process inputs and outputs

​FAQs

​How do I stop large traces from being truncated?

​How do I disable tracing?

​Environment variable

​Client initialization

​Context manager

​How do I capture information about a Call?

Creating Calls

1. Automatic tracking of LLM libraries

Summary

2. Decorating and wrapping functions

Trace sync & async generator functions

Getting a handle to the call object during execution

Call display name

Attributes

Trace parallel (multi-threaded) function calls

3. Manual Call tracking

4. Track class and object methods

Viewing Calls

Updating Calls

Set display name

Add feedback

Delete a Call

Delete multiple Calls

Querying and exporting Calls

Call schema

Saved views

View a W&B run in the Traces table

Python example

Configure autopatching

Disable all autopatching

Disable a specific integration

Post-process inputs and outputs

FAQs

How do I stop large traces from being truncated?

How do I disable tracing?

Environment variable

Client initialization

Context manager

How do I capture information about a Call?